The present invention relates to a television signal processor for generating a video signal in wide dynamic range, a television camera using the same, and a method for television signal processing. The wide dynamic range television camera is designed to properly reproduce any images even though different-brightness images are mixed in an object being shot.
The conventional, general television camera has a limit of about fourfold dynamic range because of limited charge storage capacity of the imaging device. Therefore, as when shooting an objet including both a dark indoor image and a bright outdoor image, or a scene including a mixture of an extremely bright (high-brightness) object (outdoor) and a relatively dark (low-brightness) object (indoor), we cannot help controlling exposure time or the like so that only one of the high-brightness objet and low-brightness object can be obtained to have a correct level. Making the low-brightness portion have a correct level will cause the high-brightness portion white saturation, while making the high-brightness portion have a correct level will cause the low-brightness portion black level cut-off.
Recently, a wide dynamic range imaging device has been developed so as to solve the above problems. As shown in FIG. 7, charge accumulating operation is performed for the normal exposure time during the vertical video period, and for a short exposure time during the vertical blanking period to produce a standard luminance video signal (long-time exposure video signal) V1 by which a standard-brightness object can be reproduced to have a correct level according to the normal exposure and a high-brightness video signal (short-time exposure video signal) V2 by which an extremely bright object can be reproduced to have a correct level according to the short exposure. In addition, a dual amplification system has been developed. As shown in FIG. 8, a video signal C from an imager 22 is supplied to amplifiers 23, 24 having different amplification factors so that the standard-brightness video signal V1 and high-brightness video signal V2 can be produced from the amplifiers.
These standard-brightness video signal V1 and high-brightness video signal V2 produced by the wide dynamic range imager or dual amplification system are, as shown in FIG. 9, supplied to multipliers 26, 27 of which the multiplication factors L, S are calculated by a multiplication factor calculator 25 according to a combining ratio R which will be described later. The multiplied signals are supplied to a combiner 28 where they are combined to produce a wide dynamic range video signal W about 64 times wider than the normal one.
The video signal produced from the imagers of the television camera are subjected to nonlinear processing such as white balance, gamma correction and knee characteristic correction, and then supplied to the outside. The wide dynamic range television camera generates two or more different video signals for one object, and adequately combines that plurality of signals to produce a wide dynamic range video signal. Those plurality of video signals can be produced by changing the exposure condition or gain at the time of shooting as will be described later.
The present invention tries to provide a high-quality video signal by solving the problems that, as described below, are caused by the video signal processing in the wide dynamic range television camera, such as in the white balance process, nonlinear process or signal combining process.
The above television camera that produces a wide dynamic range video signal generates, from its one solid-state imager, two video signals: a long-time exposure video signal with the exposure condition meeting the low-brightness region of an object, and a short-time exposure video signal with the exposure condition meeting the high-brightness region of the object. These long-time exposure and short-time exposure signals are subjected to a certain combining process, and then to white balance process in a video signal processor.
The construction of this conventional television camera will be described with reference to FIG. 5. Referring to FIG. 5, there is shown a charge coupled device (CCD) 11 as a solid state imager for generating a video signal by transducing the light image from an object into a charge image. The imager 11 generates the short-time exposure video signal by, for example, exposure to light for {fraction (1/2000)} second, and the long-time exposure video signal by exposure to light for a longer time than the short-time exposure video signal, for example, exposure to light for {fraction (1/60)} second, with their time base compressed during one horizontal scanning period of the video signal produced from the television camera 10. There are also shown a sample and hold/automatic gain controller (CDS and AGC) 12 that samples and holds the video signal from the CCD 11 and amplifies it to a necessary level, and an A/D converter 13 for converting the analog video signal from the CDS and AGC 12 to a digital video signal. Shown at 14 is a synchronizer for separating the long-time exposure and short-time exposure video signals of digital video signals in each horizontal scanning period from the A/D converter 13, expands the their time base to one horizontal period and synchronizes those separated and expanded video signals so that they can be produced at a time. Shown at 17 is a combiner that receives the long-time exposure and short-time exposure video signals from the synchronizer 14, and combines those signals by a predetermined way. In this method for combining, the two input signals are combined to produce such a combined video signal as to have a video signal level corresponding to, for example, the level of a more appropriate-exposure one of the two video signals. As a result of such combining process, for example, the region in which the details are not clear because the long-time exposure video signal shows white saturation on the screen is made appropriate by the correct signal level of the short-time exposure video signal, and the region in which the details are not clear because the short-time exposure video signal shows black level cut-off is made appropriate by the correct signal level of the long-time exposure video signal. The combined video signal from the combiner 17 is supplied to a digital signal processor (DSP) 15. In this DSP 15, the combined video signal is subjected to a certain video treatment for gamma correction and white balance. The processed video signal is supplied from the DSP 15 to the subsequent stage (not shown).
FIG. 6 is a block diagram of one example of the white balance adjuster. This white balance adjuster can be constructed by a DSP or a combination of circuits having the functions of the blocks shown. An RGB gain processor 30 adjusts individually the gains of the amplitude levels of the color signals R (red), G (green) and B (blue). A color difference conversion processor 31 generates color difference signals R-Y, B-Y from the R, G, B signals and Y (luminance) signal. A white signal component detector 32 detects and produces the white signal component included in a certain white detection level frame range from the color difference signals R-Y, B-Y and Y signal. A white balance controller 33 detects in what color direction and how much the white balance is deviated by comparing the white signal components of R-Y and B-Y, and produces a signal WB (corresponding to the color temperature) indicating the deviation of the white balance. A gain controller 34 adjusts the gains of the R signal and B signal in the RGB gain processor 30 in such a direction that the WB becomes zero according to the deviation signal WB.
The object to be shot includes, as shown in FIG. 2, both a human FIG. 20 in a room under fluorescent lighting, and a sunny outdoor scene 21 seen through a window. The room is a dark region (low brightness) as hatched, and the outdoor scene 21 is a much brighter region (high brightness) than the room. Since the light source in the room is fluorescent lighting, and the outdoor scene is under sunlight, the color temperatures are much different in the indoor and outdoor images.
The problems occurring when the conventional wide dynamic range camera makes white balance will be described below. When the object shown in FIG. 2 is shot by the conventional wide dynamic range television camera shown in FIG. 5, two video signals can be produced: an indoor video signal with an appropriate low-brightness exposure level and an outdoor video signal with an appropriate high-brightness exposure level. In other words, the low-brightness image can be shot for a long-time exposure to produce a long-time exposure video signal with an appropriate level, and the high-brightness image can be shot for a short-time exposure to produce a short-time exposure video signal with an appropriate level. The long-time exposure video signal can be produced as an accumulated charge signal in each vertical scanning period from the CCD, and the short-time exposure video signal as an accumulated charge signal in each vertical blanking period from the CCD.
The two video signals are combined by the combiner 17 to produce one video signal. This video signal experiences white balance process in DSP 15. At this time, white level reference for white balance is determined on the basis of an appropriate one of the long-time exposure video signal component with the exposure condition matched to the indoor image or the short-time exposure video signal component with the exposure condition matched to the outdoor image. Therefore, the white balance adjusted for one video signal component becomes correct, but the white balance for the other video signal component of different color temperature is not correct, with the result that either long-time exposure video signal component or short-time exposure video signal component shows unnatural color on the monitor. In addition, when white balance is adjusted so as to match the intermediate color temperature between the long-time and short-time exposure signal components, reproduction of both indoor and outdoor images shows unnatural color.
The present invention is to solve these problems by forcing the chrominance signals of the standard-brightness and high-brightness video signals to be subjected to a predetermined process and to the combining process after the white balance treatment, thereby producing a wide dynamic range video signal with no loss and with an optimum level.
The difficulties in the signal combining process will be described, and in this case the conventional signal combining processor shown in FIG. 9 is used to treat the video signals from a wide dynamic range television camera.
If the maximum combining ratio R of high-brightness video signal V2 occupying part of the obtained wide dynamic range video signal W is represented by R%, the multiplication factor of standard-brightness video signal V1 by L, and the multiplication factor of high-brightness video signal V2 by S, the multiplication factors L, S are given as fixed factors by the following expressions.
L=(100%xe2x88x92R%)/100%
S=R%/100%
From the above expressions it will be understood that L+S=1. Even if both standard-brightness video signal V1 and high-brightness video signal V2 fed to the combiner have the maximum input level of 100%, the signal level after combining is 100%xc3x97L+100%xc3x97S=100%, and thus the wide dynamic range video signal W can be confined within 100% level.
If the signal level after combining exceeds 100%, the signal component over 100% is compressed up to 100%, or white saturation since the final output of the camera is limited to 100%.
Let us consider the case in which the combining ratio R=50, or the standard-brightness video signal V1 and high-brightness video signal V2 are combined at a ratio of 50%:50%, namely L=0.5, S=0.5. As is obvious from FIG. 13, when the peak value P of high-brightness video signal V2 is smaller than the saturation level 100% of the video signal, for example, 60%, the amplitude of the wide dynamic range video signal W after combining is 100%xc3x97L(0.5)+60%xc3x97S(0.5)=80%, and thus it does not swing up to the saturation level 100% of the video signal. Accordingly, since the output level 100% of the video signals cannot be made full use of, a low-contrast video image is reproduced.
The present invention is to eliminate these defects by providing a wide dynamic range camera capable of producing a combined wide dynamic range video signal with no loss and with an optimum level.
U.S. Pat. No. 5,589,880 filed Dec. 31, 1996 and granted Tsukui discloses a television camera using two imagers for one object to generate two different video signals from which a wide dynamic range video signal is produced.
JP-A-4-354277 discloses a television camera using a combination of a high-speed shutter, a low-speed shutter and one CCD imager to generate two video signals of different exposure times and to select and produce an appropriate one.
U.S. Pat. No. 5,420,635 discloses a television camera using an optical splitter to split the light from an object into two beams of different amounts of light and using two CCD imagers to receive these beams and generate two video signals.
U.S. Pat. No. 5,455,621 discloses an imaging apparatus which produces wide dynamic range video signals reducing signal saturation and noises within dark image and bright image by processing two video signals having different charging times obtained from an imaging device capable of varying charging time.
It is an object of the invention to provide a television signal processor for a wide dynamic range television camera capable of proper white balance for any region of one object including image regions of even different color temperatures of light source so that video signals can be generated with excellent color reproduction, a method for signal processing, and the wide dynamic range television camera.
It is another object of the invention to provide a television signal processor for a wide dynamic range television camera capable of properly combining a standard-brightness video signal and high-brightness video signal subjected to white balance process so that appropriate white balance can be made for any region of the shot image, a method for signal processing, and the wide dynamic range television camera.
It is still another object of the invention to provide a television signal processor for a wide dynamic range television camera capable of properly combining a standard-brightness video signal and high-brightness video signal to produce a wide dynamic range video signal with no loss and with an optimum level, a method for signal processing, and the wide dynamic range television camera.
According to an aspect of the invention, there is provided a television signal processor that generates a wide dynamic range video signal on the basis of a first video signal matched to the brightness of a low-brightness region of an object image, and a second video signal matched to the brightness of a high-brightness region of the object image, and that has first adjusting means for subjecting the first video signal to white balance process on the basis of the first video signal to produce a white-balanced video signal, second adjusting means for subjecting the second video signal to white balance process on the basis of the second video signal to produce a white-balanced video signal, and combining means for combining the outputs from the first and second adjusting means.
According to another aspect of the invention, there is provided a television signal processor that generates a wide dynamic range video signal on the basis of a first video signal matched to the brightness of a low-brightness region of an object image, and a second video signal matched to the brightness of a high-brightness region of the object image, and that has means for detecting the peak value of the second video signal, means for determining a combining ratio of the first and second video signals according to the detected peak value, and combining means for combining the first and second video signals at the combining ratio.
According to another aspect of the invention, there is provided a television signal processor that generates a wide dynamic range video signal on the basis of a first video signal matched to the brightness of a low-brightness region of an object image, and a second video signal matched to the brightness of a high-brightness region of the object image, and that has means for detecting histogram values of particular signal level regions of the first and second video signals, means for nonlinear-processing the first and second video signals according to the detected histogram values, and combining means for combining the nonlinear-processed first and second video signals at a predetermined ratio.
According to another aspect of the invention, there is provided a television signal processor that generates a wide dynamic range video signal on the basis of a first video signal matched to the brightness of a low-brightness region of an object image, and a second video signal matched to the brightness of a high-brightness region of the object image, and that has means for detecting a histogram value of a particular signal level region of the second video signal, means for determining a combining ratio of the first and second video signals according to the detected histogram value, means for detecting the peak value of the second video signal, means for nonlinear-processing the first video signal according to the detected peak value and the combining ratio, and combining means for combining the nonlinear-processed first video signal and the second video signal at the determined combining ratio.
According to still another aspect of the invention, there is provided a television signal processor that generates a wide dynamic range video signal on the basis of a first video signal matched to the brightness of a low-brightness region of an object image, and a second video signal matched to the brightness of a high-brightness region of the object image, and that has first adjusting means for subjecting the first video signal to white balance process on the basis of the chrominance signal component and luminance signal component of the first video signal to produce white-balanced chrominance signal component and luminance signal component, second adjusting means for subjecting the second video signal to white balance process on the basis of the chrominance signal component and luminance signal component of the second video signal to produce white-balanced chrominance signal component and luminance signal component, means for determining a first weighting coefficient for the chrominance signal component of the first video signal according to the luminance signal component level of the first video signal, means for determining a second weighting coefficient for the chrominance signal component of the second video signal according to the luminance signal component level of the second video signal, first multiplying means for multiplying the chrominance signal component from the first adjusting means by the first weighting coefficient, second multiplying means for multiplying the chrominance signal component from the second adjusting means by the second weighting coefficient, first combining means for combining the luminance signal components from the first and second adjusting means, and second combining means for combining the chrominance signal components from the first and second multiplying means.
In addition, according to the invention, there are provided television cameras that generate a wide dynamic range video signal, and that have a television signal processor according to any one of the above aspects of the invention, and imaging means for generating the first video signal matched to the brightness of a low-brightness region of an object image, and the second video signal matched to the brightness of a high-brightness region of the object.
Moreover, according to the invention, there are provided television signal processing methods concerned with the above television signal processors according to the invention.